Appl. No. 09/927,558 

Amdt dated 2/07/2005 

Reply to the Office Action of 10/07/2004 



REMARKS/ARGUMENTS 

Reexamination and reconsideration of this application as amended is requested. By this 
amendment, Claims 1-3, 10-12, 18-20 and 29, have been amended. After this amendment, 
Claims 1-29 remain pending in this application. 

Drawing Objections 

(1) The Examiner objected to FIGs. 4, 9A-9B, 11, and 12, for various informalities including 
hand-drawn labels, captions, diagrams, and charts. Applicant has submitted herewith twelve 
sheets of formal drawings with all sheets being labeled "Replacement Sheet". In the submitted 
formal drawings, Applicant has amended FIGs. 4, 9A-9B, 11, and 12, to correct the various 
informalities indicated by the Examiner, and Applicant believes that the objection has been 
overcome. Applicant requests that the Examiner withdraw the objection to the drawings. 

Claim Objections 

(2) The Examiner has objected to Claim 29 because of a minor informality. Applicant has 
amended Claim 29 to replace "wxw " with "coxco" to correct the very minor informality and not 
for patentability or to further limit the claim in view of any prior art. Applicant therefore 
believes that the objection to Claim 29 has been overcome. In view of the amendment and 
remarks above, Applicant kindly requests that the Examiner withdraw the objection to Claim 29. 
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Claim Rejections - 35 USC S 112 

(3-7) The Examiner rejected Claims 2-3, 11, and 19-20 under 35 U.S.C. 112, second 
paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject 
matter which Applicant regards as the invention. 

Applicant has amended Claims 2 and 19 to more clearly and affirmatively recite the 

present invention. Dependent Claims 3 and 20, that depend from Claims 2 and 19, were 

amended to conform to the amendments of respective claims 2 and 19. Applicant specifically 

has deleted the following language from amended Claims 2 and 19: 

"determining a multiple-view correspondence between the plurality of detected 
image pixels in the first image and the plurality of detected image pixels in the at 
least a second image, the multiple-view correspondence being a refinement of the 
first two-view correspondence, resulting in a second potential match set of 
candidate image pixels between the first image and the at least a second image, 
wherein the second potential match set is based at least in part on a computation 
of reprojection error for matched pixels that resulted from a projective 
reconstruction of the second potential match set". 

Further, Applicant has amended the respective claim language for these claims to refer to 
a first multiple- view potential match set and to a second multiple- view potential match set. This 
now more clearly and definitely recites the new and novel features of the presently claimed 
invention. For example, amended Claim 2 now more clearly and distinctly recites "[a] potential 
two-view match set of candidate image pixels between the first image and the one of the at least 
a second image; concatenating a plurality of two-view match sets to form a first multiple- 
view potential match set; and determining a second multiple-view potential match set that 
is a refinement of the first multiple-view potential match set". (Emphasis added). Support 
for these amendments may be found in the specification as originally filed. See generally the 
section beginning on page 12 of the Specification, and also see paragraphs [0031]-[0058] of 
U.S. Patent Application Publication No. 2002/0172413 Al, the teachings of which being 
incorporated by reference into the present patent application. No new matter was added by the 
amendment. 
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Applicant believes that in view of the amendment and remarks above the rejection of Claims 2-3 
and 19-20 under 35 U.S.C. 112, second paragraph, has been overcome. Applicant kindly 
requests that the Examiner withdraw the rejection of Claims 2-3 and 19-20. 

Additionally, Applicant has amended Claim 10 to more clearly and affirmatively recite 
the present invention. Amended Claim 10 more clearly and affirmatively recites "a second 
image window of the same pre-determined size in one of at least one reference view". 
Therefore, Claim 10 now provides proper antecedent basis for amended Claim 11. Support for 
these amendments may be found in the specification as originally filed, see for example page 
13, lines 3-5 and 10-13, and page 29, lines 15-16. No new matter was added by the amendment. 

Applicant has amended Claim 1 1 to more clearly and affirmatively recite the present 
invention. Amended Claim 1 1 more clearly and affirmatively recites "rectification of at least 
one pair of images corresponding to the base view of the scene and one of the at least one 
reference view". See for example page 13, lines 3-5 and 10-13, and page 29, lines 15-16. No 
new matter was added 

In view of the amendment to Claims 2-3, 10-11, and 19-20, and the remarks above, 
Applicant believes that the rejection of Claims 2-3, 11, and 19-20, under 35 U.S.C. § 112, 
second paragraph, as discussed above, has been overcome. Applicant respectfully requests that 
the Examiner withdraw the rejection of Claims 2-3, 1 1, and 19-20. 

(8-10) The Examiner rejected Claim 29 under 35 U.S.C. 1 12, first paragraph, as failing to 
comply with the enablement requirement. 

Applicant has amended the Specification at page 4, line 15 in conformance to the 
language already found in Claim 29 to include "In Equation (7) the E g is calculated using finite 

difference calculations." No new matter was added. 
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Further, "finite difference" as an approximation to derivatives is a well known term for 
those of ordinary skill in the art. For example, "The ability to expand smooth functions in Taylor 
series allows the values of derivatives to be approximated, rather than calculated analytically." 
See Philip E. Gill et al., Practical Optimization , Page 54, Academic Press; attached hereto as 
Exhibit 1. Also, please refer to the following URL, 

http://mathworld.wolfram.com/FiniteDifference.html, as further evidence of the above. 

In view of the amendment to the specification and the remarks above, Applicant believes 
that the rejection of Claim 29 under 35 U.S.C. § 112, first paragraph, has been overcome. 
Applicant requests that the Examiner withdraw the rejection of Claim 29. 

Claim Rejections - 35 USC § 102 

(11-26) The Examiner rejected Claims 1, 4-9, 12-18, and 21-26 under 35 U.S.C. 102(b) as being 
anticipated by Chen and Medioni, "A Volumetric Stereo Matching Method: Application to 
Image-Based Modeling"' IEEE 19999. This rejection is respectfully traversed. 

Applicant has amended independent Claims 1 and 1 8 to more clearly and distinctly recite 
the present invention. Applicant has amended Claims 1 and 1 8 specifically to more clearly recite 
"wherein the first view is one of at least three views of a scene"; "a plurality of images, each 
image representing one of the at least three views of the scene, wherein each of the at least three 
views of the scene are situated in a non-linear arrangement and are further oriented in a 
plurality of non-parallel planes relative to each other". Amended Claim 12 recites similar 
claim language. Support for these amendments may be found in the specification and drawings 
as originally filed. For example, see FIG. 3; and see the specification, on page 7, lines 5-13. No 
new matter was added by the amendment. 

Chen [VSMM99] teaches a volumetric stereo matching method that is applied to image- 
based modeling. Two cameras, which are not parallel, are used to take the stereo images. 
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However, Chen [VSMM99] teaches that the task of rectification is to numerically align the two 
image planes so that they are coplanar and their scanlines are horizontally coincident. 

In contrast, as now recited for amended Claims 1 and 18, and similarly for amended 
Claim 11, the presently claimed invention recites detecting a plurality of pixels in a base image 
that represents a first view of a scene. The first view is one of at least three views of a scene. A 
3-D depth of the plurality of pixels in the base image is determined by matching correspondence 
to a plurality of pixels in a plurality of images. Each image represents one of the at least three 
views of the scene that are situated in a non-linear arrangement and are further oriented in a 
plurality of non-parallel planes relative to each other. The present invention further recites 
tracing pixels in a virtual piecewise continuous depth surface by spatial propagation starting 
from the detected pixels in the base image by using the matching and corresponding plurality of 
pixels in the plurality of images. As a result, the virtual piecewise continuous depth surface 
viewed from the base image, each successfully traced pixel being associated with a depth in the 
scene viewed from the base image is created. 

The present invention, as now recited for amended Claim 12, similarly recites the above 
claim language with respect to the "views of a scene" and the arrangement and orientation of the 
views of the scene. 

Chen [VSMM99] does not teach, anticipate, or suggest, at least three views of a scene 
nor does the reference teach that each image represents one of the at least three views of the 
scene that are situated in a non-linear arrangement and are further oriented in a plurality of non- 
parallel planes relative to each other. In fact, Chen [VSMM99] teaches that images taken by two 
cameras are numerically aligned so that the images are coplanar and their scanlines are 
horizontally coincident in order to perform the method as taught by Chen [VSMM99]. See 
Chen [VSMM99] page 30 under the heading 2 (Image Rectification). 

On the other hand, the present invention as recited by amended Claims 1,12, and 18, may 
perform the method as recited on images that are non-parallel and non-linear. Therefore, in view 
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of the amendments and remarks above, Applicant believes that since Chen [VSMM99] does not 
teach, anticipate, or suggest, inter alia, the presently claimed "at least three views of a scene", 
and further does not teach, anticipate, or suggest, the presently claimed "each image represents 
one of the at least three views of the scene that are situated in a non-linear arrangement and are 
further oriented in a plurality of non-parallel planes relative to each other", the rejection of 
Claims 1, 12 and 18, under 35 U.S.C. 102(b) as being anticipated by Chen [VSMM99] has been 
overcome. The Examiner should withdraw the rejection of these claims. 

Claims 4-9, 13-17, and 21-26, depend from Claims 1, 12, and 18, respectively, either 
directly or by way of an intervening claim, and since dependent claims recite all of the 
limitations of the independent claim; it is believed that, therefore, claims 4-9, 13-17, and 21-26 
also recite in allowable form. Therefore, Applicant believes that the rejection of Claims 4-9, 13- 
17, and 21-26, under 35 U.S.C. 102(b) as being anticipated by Chen [VSMM99] has also been 
overcome. The Examiner should withdraw the rejection of these claims. 

Claim Rejections - 35 USC § 103 

(27-30) The Examiner rejected Claims 27-28 under 35 U.S.C. 103(a) as being unpatentable 
over Chen and Medioni, "A Volumetric Stereo Matching Method: Application to Image-Based 
Modeling"' IEEE 19999. This rejection is respectfully traversed. 

Claims 27-28 depend from amended Claim 18. The above arguments and remarks 

regarding Claims 1 and 18, and more specifically with respect to the claim language 

"detecting a plurality of pixels in a base image that represents a first view of a scene, 
wherein the first view is one of at least three views of a scene; determining 3-D depth of 
the plurality of pixels in the base image by matching correspondence to a plurality of 
pixels in a plurality of images, each image representing one of the at least three views 
of the scene, wherein each of the at least three views of the scene are situated in a 
non-linear arrangement and are further oriented in a plurality of non-parallel 
planes relative to each other", 
are likewise applicable here in support of the allowability of dependent Claims 27-28. These 

applicable arguments have already been presented above and will not be repeated here. 
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Accordingly, in view of the amendments and remarks above, since Chen [VSMM99] 
does not teach, anticipate, or suggest, the presently claimed invention as recited for amended 
Claim 18, and for dependent Claims 26-27, Applicant believes that the rejection of Claims 27-28 
under 35 U.S.C. 103(a) has been overcome. The Examiner should withdraw the rejection of 
these claims. 

(31-39) The Examiner rejected Claims 2-3 and 19-20 under 35 U.S.C. 103(a) as being 
unpatentable over Chen and Medioni, "A Volumetric Stereo Matching Method: Application to 
Image-Based Modeling'" IEEE 19999 in view of Zhang, Deriche, Faugeras, and Luong, "A 
Robust Technique for Matching Two Uncalibrated Images Through the Recovery of Unknown 
Epipolar Geometry", INRIA 1994). This rejection is respectfully traversed. 

Applicant has amended Claims 2 and 19 to more clearly and distinctly recite the present 
invention. Amended Claim 2, as representative for these amended claims, more clearly and 
distinctly recites "[a] potential two-view match set of candidate image pixels between the first 
image and the one of the at least a second image; concatenating a plurality of two-view match 
sets to form a first multiple-view potential match set; and determining a second multiple- 
view potential match set that is a refinement of the first multiple-view potential match set". 
Support for these amendments may be found in the specification as originally filed. See 
generally the section beginning on page 12 of the Specification, and also see paragraphs [0031]- 
[0058] of U.S. Patent Application Publication No. 2002/0172413 Al that are incorporated by 
reference in the present patent application. No new matter was added by this amendment. 

First of all, Claims 2-3, and 19-20 depend from amended independent Claims 1 and 18, 
respectively, and as discussed above, since dependent claims recite all of the limitations of the 
independent claim, it is believed that, therefore, Claims 2-3 and 19-20 are not taught, anticipated, 
or suggested by the teachings of Chen [VSMM99]. Additional arguments regarding the 
allowability of Claims 2-3, and 19-20 are given below. 
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As discussed above, Chen [VSMM9] teaches a volumetric stereo matching method that is 
applied to image-based modeling using two non-parallel cameras. The images being numerically 
aligned so that the images are coplanar and their scanlines are horizontally coincident. 

Zhang teaches a robust technique for matching two uncalibrated images through the 
recovery of the unknown epipolar geometry. The Examiner directs Applicant to Section 6.3 on 
pages 16-19 of Zhang, wherein Zhang teaches a stereo matching method. Additionally, Zhang 
teaches that outliers will severely affect the precision of the fundamental Matrix taught by 
Zhang. Therefore, possible outliers should be taken into account in the initial correspondences. 

In contrast, as recited for amended Claims 2 and 19, the presently claimed invention 
detects a plurality of image pixels in a first image corresponding to a first view of a scene. A 
plurality of image pixels in at least a second image corresponding to a respective at least a 
second view of the scene is detected. The at least a second image deviates from the first image 
as a result of camera relative motion. A first two-view correspondence between the plurality of 
detected image pixels in the first image and a plurality of detected image pixels in one of the at 
least a second image is determined and results in a potential two-view match set of candidate 
image pixels between the first image and the one of the at least a second image. A plurality of 
two-view match sets is concatenated to form a first multiple-view potential match set and a 
second multiple-view potential match set that is a refinement of the first multiple-view 
potential match set is determined. 

Chen [VSMM99] does not teach, anticipate, or suggest, "a plurality of two-view match 
sets is concatenated to form a first multiple-view potential match set and a second multiple-view 
potential match set that is a refinement of the first multiple-view potential match set is 
determined". As stated above, Chen [VSMM99] teaches two cameras, and therefore, two views 
of an image. See the Chen [VSMM99] specification, for example, on page 30, Section 2. 
Therefore, Chen [VSMM99] does not teach or suggest the present invention, as now recited for 
amended Claims 2 and 19. 
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Turning now to Zhang, Zhang also does not teach, anticipate, or suggest "a plurality of 
two-view match sets is concatenated to form a first multiple-view potential match set and a 
second multiple-view potential match set that is a refinement of the first multiple-view potential 
match set is determined". In fact, Zhang explicitly teaches two views of an image. See for 
example page 6, Section 3, and FIG. 1 . 

Furthermore, Zhang does not teach or suggest at least three views of a scene nor does the 
reference teach or suggest that each image represents one of the at least three views of the scene 
that are situated in a non-linear arrangement and are further oriented in a plurality of non-parallel 
planes relative to each other. See for example FIG. 1 . Therefore, Zhang does not teach or 
suggest the presently claimed invention as recited for amended Claims 2 and 1 9, or for amended 
Claims 1 and 18 from which Claims 2 and 19 depend. 

Applicant has amended Claims 3 and 20 to conform to the amendments of amended 
Claims 2 and 19, from which Claims 3 and 20 depend. Amended Claims 3 and 20 more clearly 
and distinctly recites "wherein the second multiple-view potential match set is based at least in 
part on a least median of squares computation of the reprojection errors related to matched pixels 
in the first multiple-view potential match set." 

The above arguments and remarks regarding Chen [VSMM99] and Zhang with respect to 
Claims 2 and 19, and more specifically, a "first multiple-view potential match set" and a "second 
mulitple-view potential match set", are likewise applicable here in support of the allowability of 
Claims 3 and 20. 

Accordingly, in view of the amendments and remarks above, since neither Chen 
[VSMM99], Zhang, nor any combination thereof, teaches, anticipates, or suggests, the presently 
claimed invention as recited for amended Claims 2-3 and 19-20, Applicant believes that the 
rejection of Claim 2-3 and 19-20 under 35 U.S.C. 103(a) has been overcome. The Examiner 
should withdraw the rejection of these claims. 
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(40-45) The Examiner rejected Claims 10-11 under 35 U.S. C. 103(a) as being unpatentable over 
Chen and Medioni, "A Volumetric Stereo Matching Method: Application to Image-Based 
Modeling"' IEEE 19999 in view of Okutomi and Kanade, "A Multiple-Baseline Stereo", IEEE 
1993 in further view of Lewis, "Fast normalized Cross-Correlation", 1995. This rejection is 
respectfully traversed. 

First of all, Claims 10-11 depend from amended independent Claim 1 by way of an 
intervening claim, and as discussed above, since dependent claims recite all of the limitations of 
the independent claim, it is believed that, therefore, Claim 1 and its dependent Claims 10-11 are 
not taught, anticipated, or suggested by the teachings of Chen [VSMM99]. Additional 
arguments regarding the allowability of Claims 10-11 are given below. 

Okutomi teaches a stereo matching method that uses multiple stereo pairs with various 
baselines to obtain preceise distance estimates without suffering from ambiguity. The Examiner 
directs Applicant to the Abstract, page 353-355, and page 362 wherein Okutomi teaches that the 
summation of the sum of squared differences (SSD) from multiple stereo pairs can be used to 
indicate the "correctness" of a set of matching points. The Examiner states that this summation is 
analagous to a matching cost in that it approaches a minimum for correct matches. Okutomi 
teaches the computation is made within a predetermined window W belonging to each of the 
stereo images. 

In contrast, the present invention, as recited for amended Claim 1 from which Claims 10- 
1 1 depend from by way of an intervening claim, recites among other things that "each image 
represents one of the at least three views of the scene that are situated in a non-linear 
arrangement and are further oriented in a plurality of non-parallel planes relative to each other". 

Okutomi does not teach or suggest that each image represents one of the at least three 
views of the scene that are situated in a non-linear arrangement and are further oriented in a 
plurality of non-parallel planes relative to each other. In fact, Okutomi explicitly teaches that 
different baselines are generated by a lateral displacement of a camera. See for example the 
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Abstract. Therefore, Okutomi does not teach or suggest the presently claimed invention as now 
recited by amended Claim 1 and by dependent Claims 10-11. 

Lewis teaches how unnormalized cross correlation can be efficiently normalized using 
precomputing integrals of the image and image 2 over the search window. However, nowhere 
does Lewis teach detecting a plurality of pixels in a base image that represents a first view of a 
scene, wherein the first view is one of at least three views of a scene; determining 3-D depth of 
the plurality of pixels in the base image, by matching correspondence to a plurality of pixels in a 
plurality of images, each image representing one of the at least three views of the scene, wherein 
each of the at least three views of the scene are situated in a non-linear arrangement and are 
further oriented in a plurality of non-parallel planes relative to each other; and tracing pixels in a 
virtual piecewise continuous depth surface by spatial propagation starting from the detected 
pixels in the base image by using the matching and corresponding plurality of pixels in the 
plurality of images to create the virtual piecewise continuous depth surface viewed from the base 
image, each successfully traced pixel being associated with a depth in the scene viewed from the 
base image, as recited for amended Claim 1 from which Claims 10-11 depend. 

Accordingly, in view of the amendments and remarks above, since neither Chen 
[VSMM99], Okutomi, Lewis, nor any combination thereof, teaches, anticipates, or suggests, the 
presently claimed invention as recited for Claims 10-11, Applicant believes that the rejection of 
Claim 10-11 under 35 U.S.C. 103(a) has been overcome. The Examiner should withdraw the 
rejection of these claims. 

Allowable/Allowed Subject Matter 

(46-47) The Examiner stated that Claim 29 would be allowable if rewritten or amended to 
overcome the rejeciton(s) under 35 U.S.C. 112, first paragraph, set forth above. 

As discussed above in section (2), Applicant has amended Claim 29 to overcome the 35 
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U.S.C. 112, first paragraph rejection, and Applicant believes that Claim 29 now recites in 
allowable form. Applicant kindly requests that the Examiner allow Claim 29. 

Citation of Relevant Prior Art 

(48-49) Applicant has reviewed the prior art made of record and believes that each of the cited 
references alone or in any combination, and including any combination with Chen [VSMM9], 
Zhang, Okutomi, and/or Lewis, does not teach, anticipate, or suggest, the presently claimed 
invention. 

Conclusion 

The foregoing is submitted as full and complete response to the Official Action mailed 
October 7, 2004, and it is submitted that Claims 1-29 are in condition for allowance. 
Reconsideration of the rejection is requested. Allowance of Claims 1-29 is earnestly solicited. 

No amendment made was related to the statutory requirements of patentability unless 
expressly stated herein. No amendment made was for the purpose of narrowing the scope of any 
claim, unless Applicant has argued herein that such amendment was made to distinguish over a 
particular reference or combination of references, 

Applicant acknowledges the continuing duty of candor and good faith to disclosure of 
information known to be material to the examination of this application. In accordance with 37 
CFR §§ 1.56, all such information is dutifully made of record. The foreseeable equivalents of 
any territory surrendered by amendment are limited to the territory taught by the information of 
record. No other territory afforded by the doctrine of equivalents is knowingly surrendered and 
everything else is unforeseeable at the time of this amendment by the Applicant and the 
attorneys. 
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The present application, after entry of this amendment, comprises twenty-nine (29) 
claims, including four (4) independent claims. Applicant has previously paid for twenty-nine 
(29) claims including four (4) independent claims. Applicant, therefore, believes that a fee for 
claims amendment is currently not due. 



However, a petition for extension of time to file this Response has been attached. The 
Commissioner is authorized to charge the extension fee for response within the first month of 
$120 , or if this fee amount is insufficient or incorrect, then the Commissioner is authorized to 
charge the appropriate fee amount to prevent this application from becoming abandoned to 
Deposit Account 50-1556. 



If the Examiner believes that there are any informalities that can be corrected by 
Examiner's amendment, or that in any way it would help expedite the prosecution of the 
patent application, a telephone call to the undersigned at (561) 989-9811 is respectfully 
solicited. 



The Commissioner is hereby authorized to charge any fees that may be required or credit 
any overpayment to Deposit Account 50-1556. 
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In view of the preceding discussion, it is submitted that the claims are in condition for 
allowance. Reconsideration and re-examination is requested. 



FLEIT, KAIN, GIBBONS, GUTMAN 

BONGINI & BIANCO P.L. 

55 1 N.W. 77th Street, Suite 111 • 

Boca Raton, FL 33487 

Tel (561)989-9811 

Fax (561)989-9812 



Respectfully submitted, 





Jose Gutmao/ 
Reg. No. 35,171 
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IN THE DRAWING: 



Please replace all of the sheets of the Drawing figures with the 12 sheets identified at the top 
margin as "Replacement Sheet" and that are attached to this Amendment. These twelve (12) 
replacement sheets provide formal drawings conforming to the originally filed drawing figures. 
No new matter was added. 
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